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/ • Abstract 



Tutorial dialogues can be analyzed as an 
interaction in viiich a tutor "debugs^ •a,stud'^t's knowledge 
-representation by diagnosing and corre^cting conceptual 
misunderstandings." In this psoer, we outline some tentative 
steps toward a theory whic: descri-be.s tutorial interactions. 
We> outline the goal structure ofv>a tutor, describe types of 

conceptual bugs that students' have in their understanding o.f 

. • • \" - - ' ..^ 

• physical" processes "and- disc.uss some of the representational 

1 ' • ^ ' • ' ■ 

viewpoint^ nece.s'^ary to diagnose j^nd, correct tjiese bugs., - 



Diagnosing Students' Misconceptions in Causal Models 

• . ■ . 

w ' . Albert Stevens . 

Allan Collins . . : 
Sally Goldin 

; ■ • . . L 

We are building a computer aided • Instructional system 
which tutors students • to reason - about and understand 
physical processes. In order- to'build such* a system, we 
have- been forced to 1 confront several fundamental ^issues 
about the. tutorial pr^ces^S*.. 

(1) What is the goal struc^ture t>iat governs- a tutor's 
selesyCtion of examples, questions and statements a-t 
.? • 'different ^points in the dialogue? 



(2r) / Wi^at are the types of misconceptions that students have- 



and 'how do tutors diagnose misconceptions from errors 



students make.? ' - - r 

s ' : : ■ • -- . ^ - ■ ■ ' ■ . ; • ■ ■ ^ ■ 

(3) What ' are, the abstract ions and viewpoints that tutors 
'■ ^ ^ use to .explain physical processes? , ' 

' We "bjB-lieve we" . are - taking t-fentative steps toward a 

theory which addresses these issues. ^ ,Our approach is [to 

- -. \ ■ * " ■ ■ * 

\ ' ' ■ ■ \ ■ ' . 

work out -a theory -based on analyses of tutoring dialogues 



and experirftents a^id then^, build a' system based, on that 
itheory. Building the system .'reveals points where the theory 
is inadequate or " -wrong In s'ubsequent -iterations. ^of'-thi? 



paper r we will *descr ibe the first version of our system, its 
weak points and the steps in analysis and theory development 
we are taking to remedy these weak points. ' c '. 

■ The WHY System . ' . . ' 

'a 

The .first version of our system, called the WHY system,, 
can carry on a simp'^i-e teaching dialogue about the causes o,f 
rainfall. The theory on whi.ch the WHY system is based*' is 

formalized^ as- a set of production rules representing 

'-*■•■' ■ .. ^ ■ 

teaching ^strategies (Collins, 1977) -'anh" a script-rlike 

knowledge structure (Schank & Abel son,' 1977) ^ The script 
. u ^ ' * • - - 

structiire represents th^^ different temporal and causal^stepi 

' ' . . . ■ * u 

In processes that affect rainfall . \ ^j. Many / scripts in the 

- ■ ' * ' ■ • . *. . ■ 

systemg^can be decomposed into more'detalled subscripts. , The 

resulting embedded str uct\:ir^.4s lised^ to represent levels of 

detail of knowljed^e aboiW: different processes. C y • 

Figur4 1 illustrates"^ these^ -aspects of .knowledge 
organization. V It -^ows the^^ top-levei script-, for heavy 
rainfall which consists of , four steps Eva^porat ion ^ moy^ement , 
of tl?e air mass jf - cooling and precipitation. The subscript 
; £or the first 'step is also shown and d^Si,3ts^ 'of a more 
detailed breakdown, of the steps'^ involved rn evaporationi 



i 
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Heavy Roinfoll 



1: A warm air moss' - ' " 2: Winds carry the JThemoist'oir . / . 4: the moisture in the 
over p warm body of p^^^^^^^ warm moist air mass p^g^j^jg^ mass from over the jQ^ggg air mafs from over the 
water obsorbs a lot » from overthi bod)( ' — — > body pf water cools -a--*.body of water 



of moisture from the ' of water to over the ' over the land area precipitates over the 
body of water ^. • land mass " • land area 



I: Evaporation • • 

U: A body of water fpQjjigg , 1.2: Moisture evapo^^^^ / ' ^ 

^aterkwarm; ^ -rapidlptotheairJTass '^^^^ 1.5: The warm air mass 

• overthebodyofwater J""^^ 

' • ' ' . • '• ■ and • ^ water absorbs a lot of 

i:3:The.airmass ' .EpQ^les The warm' air mass ^...^ 

over.thebodyof ►canho|dalotof Tco^^^^ of water 

water is w(!rm . .moisture ^ ' . ' 



r 



FIGURE 1. .The script for heavy rainfall and the subscript for 
" evaporation in the first version of the Why system. 



The teaching strategy rule^ are stated^ in terms 'of a 
conditional test, of the student's response. to a question 
paired with an action . .to perform if the test is true. 
Example rules are: 

(1) if^ the student gives as an explanation a factor that 
r^B not an immediate cause in the causal chain, 

fenT ^ask for the intermediate steps. ' " 

(2) If^ the student gives as an explanation one or more 

factors that are not necessary, 
then formulate a general rule by asserting that the 
factor is necessary and ask the student if the 
rule is true . 

The theory in Coll ins. (1976) consists of twenty-four rules. 

. J ^ 

The first -Version/ of the Why system contain^; seven rules 

X 

which test for missing script steps , ask student^ questions 
and present information. 



Problems With the Current System 



L_ The current WHY system is able^ to carry on simple 
tutoring d^ialogues about tfee causes of rainfall . It can ask 
quest ion,g5/&bout places where heavy rrainfal 1 occurs, diagnose 



w 

mi:^ing s^eps in the . student's knowledge and inform the 
student about the correct steps. A bit 'of interaction with 
the system reveals several problems: (1) It has little 
global perspective ' about the dialogue and thus 'bases its 
questions and responses almost exclusively on local context. 
(2) It is sen>sitive to student errors , but typically < misses 
the cause of these errors, correcting the surface error but' 



failing to diagnose the underlying misconception that 



caixmg t:o aiagnose rne 

error reflects. (3) Threre are many important aspects cf 



physical processes and many important ways of describing 
physical processes that the WHY system fails to )use i. 

These problems paral"-?" the issues listed earlier,.^ ^e 

lack of global perspec- e -in the current system arises 

because thej^e is no coherent- .'goal structure. The 

teaching'-str^ategy rules we originally developed^ are based on 

the students' immediate responses and have no yway to 

establish and be influenced by higher leve/ ^jonls., 

* 

The types bf^^'misconcept ions in^ students ' kn4wl'edge that 
a system can diagnose are heavily dependent on the knowled^^ 
represented in the system* The script -Str'jrtures ,in the^ 

0*U] 

which rfesult because of missing substers or extra^ substeps/ 
in the various scripts. These are only tvfo cf several types 
of misconceptions that occur: We will discuss others below. 

'^^^^ The problems of failing to discuss important . aspects of,' 
pnysical " pr.ocesses , ^d ^ failing to use important ways of • 
describing physical ^ pr.OG^sses arise because the 



irren^' WHY system are ^abl^, to apresent misconceptions 



ripxy f ----- - - - 



script-subscripxy formal ism is limited.. • We ' believe that 



"1 



representing knowledge abo^t physical processes requires 
-multiple *representatioTiai viewpoints.". Our 'script 

structures provide one of- those, the viewpcJ'int of a seq^uence 

•• • 11 . ^ i^. ■ ■ ■ ■ • • 



o f tempo ralTy' o rde r ed pr bcesses , som e ca usal 1 y - r el a ted to 
others, anTd some subprocesses. of^ ,^^~T^t.hers . This 
representational viewpoint - is important, but ^^^equally 
important is the, " funct ional- viewpoint" whicfi ' emphasi zes the 
•5iinctional rel at ions^iips among " attributes of the various 
objects involved in dl fferent ^ processes . 

In the following sections,' we , wilL - discuss each of 
these problems and pV<^vide some initial ideas -about a theory 
necessary to deal -with them. It will be apparent that they_ 
are all intimately i.nter related and that- a key element 
' necessary for- their solution is an' adequate formal i-sm for 
representing the Xnow-ledgjs taught. 

Some Proposed Solu.tions - - - - 

r Goal Structuc^e . On^ of. tfie major constraints on a 
theory of tutoring -is that* it should adequately describe the 
struc.ture o.f tutor ial d ialogues • ' Our anal^yses of tutorial 
dialogues reveals a general structure' that follows from the 

J script structure . of tb^ knowledge base. Tuto.rs discuss 
topics in* a rational order, . typically following ^ the 
discussion of one process with discussion of a temporally or; 
causally adjacent |>rocess' or with a discussion of component^,, 

' subprocesses. , - / . 



er|c * ^ ' ' i§_ 



De t a V d i al og u^e s t r uc t ur e is "^^j^^uc V* more ' 
•probleiijati-cal . . -Close examination' - of. 't.utoi^f i&l. c3ialogues^ 
reyeal .th.at '-the tutor probes the stud er^t about many- 



different as.pects of the knowledge. When the student makes 
. an error, . the tutor will sometimes correct it immediately, 

' ' • • » _ . • • ■ ■».■■"* 

: *but -in many . cases . will -ask .other questions until, the 

• misconce.ption underlying the error is -isolated. The 

■ - . ■ ■ » ■-■ . 

.treatment of the misconception may then .require a> number of 
interchanges during which the -tutor tests the student's 
knowledge and supplies the .Relevant information^ We believe- 
that a powerful , perspective from which to view this more 
detailed struc.ture of tutorial dialogues is that of the 
tutor as a "debugger Much of the detailed structure of a 
dialogue results f rom ^the ^^iiitor using various strategies to 
diagnose student's' misconceptions^' or "bugs^" and then 
^.applying strategies to; correct themi 

In order to investigate this perspective of tutors as 
debugger^, 'we have conducted dialogues where the questions 
and .responses were^'communicated oyer linked terminals and 
where the tutbrs . verbally commented on two aspects of the 
dialogue as they proceeded • The two aspects were: (1) What 
they thought", the student knew or didn't know based .^on the . 
student ' s' response , ajid (2) ^ why they responded the 
student ■ in the way they did. This technique supplies data 
, normally unavailable for a — dialogue^ analysis^r providing 



'.insigrhts into how the tutor deveFops a mcJdel of the student,. 

. • ' ■ ^ . ' .. • - 

>hQ,w thd tutor organizes -the knowledge being taught and how 

' these- two -factors , influence the tutors choice' - of questions 

" « ■ ^ ■ . « 

" • ' . '. ^ r' 

and respioiise's to ther student *. ' - 

• - • ^ ^ ' ' V ' ^ ' * : - ■ v.^ - • ^ ' 

Using this ^data^ we developed, the outl lnes^-> of a theory: 

^ '*» • ■ * * " . 

of tutors' goal structures*. The-goal structure ye derived.' 

is summarized in Table .1. ■ The top level goals are (1) 

refJne^ the student's causal model' and (2) refine the 

stiid'ent's procedures for applying the model. These directly 

qoverh the selecti-on of cases. As the student's "knowledge 

becomes more refined-, moving" from an understanding . of 

first-oxder factors to 'big her- order "factors, cases, are 

■selected which are exemplary of the facto rs^ the -tutor, is 

trying to teach. As the .student^ s predictive ability- 

beeomes refined , 'cases a-re selected which are progressively^ 

more ' novel ^ and con?pl;.ex^ taxing the student's predictive 

abil ity- more and more.. . - v ' 

The* process of achieving these top-level goals involves ^ 

. ' « ■ - ' ■ / * ' r 

two types of subgoals:: diagnosis arid correction. Both of 

these subgoals gove-rn thex sel-ectio:n of basic strategies. 

. * J , : .-. ■ 

The purpose of diagnosis is to - discover - gaps and 
misconceptions in the stCident's knowledge.. ThiS' generally 
requf-res ^hat 'the ' tutor p.robe the student by asking for 
relevant factors, 'by requiring the .student to make 



Table 1 

Outlrne ^of Socratic Tutor's Go^l Structure. The manifestations refer 
the rules described^ in Collins 'C.T.976) and Stevens and' Collins (1.97^0. 



to 



Goals 
^ 



Re^^inerthe student's causal model 
mo^^i-ng from 1st to nth order factors. 



. . Manifestation- 

— -— ^ ^ ■ ■ ■ 

Case selec^tioh rules: 

Select cases tha-t. are 

exemplary of the relevent 
factor . ' 



Refine the student's* procedures' -for 
applying the causal model to novel- ; 
cases * V ' 



Case selecti'on- rules: 
•Select, less 'f am liar cases, 
exeitt|)lary . of .new factors*.' 



Subgoals 



Diagnose the student's "bugs", 
(i.e. the difference^ between the 
student's knowledge and 'the tutor's 
knowledge . ) ^ ; 

Correct the diagnosed bugs 



Ask-for- factor rules. ^ 
Prediction rules-. 
Entrapment rules. /\ ' 
Probe-reasoning-strategy rules 

In form- student rules 
"Missing-factor rules' 
Forming hypotheses rules -v/ 
Testing hypotheses rules \ 
Information-collection rules 




prledictions about carefully selected cases-/ -and by ^trying to 
ntrap the student into^ making incorrect predictions.. It Is^ 
clear from ,our analysis of hum art dialogues that diagnosis- 

• - r • * ■ ■ \ . . ■ 'f. 

cannot be completely characterized in 'terms .of a simple 

map|>drig between students J. errors and their conceptual bugs. 

-■>•■./ ■ ' , ' - ^ ■ ■ ' 

y<jj ' ' . . . •■ * 

Rather the process' involves sophisticated use of a student 

model an4 knowledge about common^ bugs in order to simulate' 

"* * - * ' • 

the student's reasoning process^sL a^f^ pinpoint the 

underlying conceptual errors or missing . info r*iat ion. In- 

some caseS/ a single answer may reveal a whole set of bugs^ 

while in other cases ^ the tutor must c'arefully^ probe the 

student, testing alternative hypothesi zed bugs ^to reveal theN 

m 1 scone ept^lo n . - ^ - . - ^ • 

typically, when a conceptual bug .is^ ^diagnosed, the 
tutor attempts to corr-ect it. This \^may require^ single 
statement for simple factual errors or an extended - dialogue 
to correct problems i^n the student's- causal mod^l. ^Iji 
Stevens and Collins (1977) we illustrate the ^^^1 icatipn of 
/ this goal structure model by analyzing^ a tutorial , dialogue . . 

Our outline of goal structure is relatively gen'eral ^ and 

■ ■ ' ■ ' ' ..... . - ^ • - - :., ... ■ _ 

probably can -be" applied tp many .different knowledge domains 

• ■ ^ ■ i ■ ■ . - , ; ::• . :■ 

and tutorial- interactiog^s. However,- in order- to specify it 
in detail, we need to know what' the bugs are , how , tfiey can " 
"be represented , . how they'' are diagnosed "from errors '311(3 how": 
.they can^be corrected. •. - < ' : 



. • Type s o f bug s • in., und e r s t and i ng r a in fa l-'l . 'Our analyse^ 
of dialogues show that, tuto'rs 'spend a good part of their 
time diagnosing conceptual bugs from' errors -Manifested in 

' * .... ♦ 

the dialogue.. We believe that one of the major skills a 
teacher "possesses is knowledge' about the types of conceptual 
bugs students" are-^likely »to have, the manifestations of 
these" bugs and methods for correcting them. It is thus 

■ •- ■ • • ■ ' ' ^ . 

cl-ear that an important component of. any teaching system is 
-a method for representing, diagnosing an,d correcting bugs. 

' To "examine the types of bugs that occur in students'^ 
.understanding of rainfall, we -carried out simple 
experiment. We compiled ;a systematic test abo.ut the causes 
of- heavy rainfall by generating questions for all major 
script node.s in' the current WHY system representation of 
rainfair:; . ThiW ^ included, for-each node,, a question which 
queried prior script steps and questions which queried 

- ' ■ X " ■ ■ • ■ ■ ■ - 

/ sub-sequent script • steps • These questions were presented to 

subjTect^. on a questionnaire. At the of the 

• ' ^ ' ** > ■ . 

,.querstionnaire • Vwas a context-setting paragraph which 
explained . that ail questions were to ., be :Snterpreted as 
referring to areas- of the world where 'heavy rainfall occurs 

■ . ■ - ■ * . 

and described wha't we .meant by- heavy rainfall ^ome typical 
questions, from this* test are: . . 
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, "How is the moisture- conten,t .. of t]ie/-,air related t.6' 

heavy rainfall?" ' / - 

.... . ■ ■ , ■ 

"What role does rising air play in causing* rainfall?" 

"What causes evaporation?" v'' • 

There were a total of 32 questions. The questions were 
initially rahdomized ^nd each sub j ectv received them in the 
same random or^der. The quest ionr^ai i*e also included two 

- questions^ one which asked the subjects to name areas of the . 

' / ■ ■ ■ 

world which have heavy rainfall and the other areas 'which -<> 
have little rainfall, ^ 

■ * • - , 'W- ' * ■ ^ . - 

The instructions /emphasized that even if the subject^^ 

■ ■ ■ ' . ^ - ' ' - . 

felt they d id -not 'know "an answer , they should try to answer 
the question. We ac3p^ted ' these instructions , because the 
typical response given- by. subjects when confropt^ed with this 
test was"""I don't, know anything about rainfall," ■.Subsequent 
probing'- revealed -^ihat they, often knew a good deal more ph.an 
they thought. The , test- was administered to ".subjects 

individually and^ typically was completed in about 30 

\. . ^ . " ■ ■ ' • ■ ■ . ^ 

^ minutes. ^ » . 

- ■ c- . ■ ' ■ . ■ ■ - 

5 ' « ^ . . ■ . ■ V 

■■ . ■ - ■>'.'■ ■ . ' ■• * . " ' 

* ■ .' 

s . . ■ 

The, exp>er iment provided us with a substantial body \of 

- data on errors and misconceptions. In order to analyze the 
responses ^in detail ^ all answers.- that were judged incorrect ■ > 
it^ere tabulat-ed . und.er the appropriate questions. . We then 

ERIC . \ . . o 
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analyired these errors by developing a t>asic set of* 

■■ - - * • ' - • 

conceptual bugs and classifying the'^errors according ""tcthis 

set.* This analysis revealed tj^o points of major interest: 

■ . ' ■ ^ •■ • . 

■ - . \ _ ' . - ^ - . ■. , ^ 

. (1) A particular conceptual bug - is often shared by 
many s|:udents. , 

(2) A particular conceptual bug is often manifested in 
many different ways. 

For example, a * bug we call the Cool iijg^-by-d^ontact bug is 
X very common # occurring for 6 of the 8- subj,ects. ^ Some 
-verbatim examples of manif estat ions^f thr^s bug are: 

' S . • ' ' 

SI) "Cold air masses cool warm air masses when they 
% . - co.llide."' . 



X 



(2) "Wind's cause air to cool . " 

(3) "Mountains "^^ause condensation because cold land 
touching warm' air causes condensation." 

(4) ' "Cold fronts, wind, snow and rain- cause air to 

cool ' , 



(53l "Cold air masses cool the clouds . so the rain 
falls." 




None of the above types of cooling are. of any 



coh^quence in causing heavy rainfalls The" type, of cooling 



1 



nefiressary 'occurs when an air- mass, is forced to rise.**. The 
rising results in expansion and energy loss. , . ^ ^ 



We identified a t'otal of sixteen ^different bugs trom .u 

" "■> , .- - ... 

this experiment.- They are "shown in order qf frequency in 
T&blfe-2. Using thes^, sixteen bugs, we were able to account J 
for -^58% of the answers origainally judged to be incorrect 
omitted. (By ignoring om l^ssions r we were able account for 
72%.) We are being conservative in this accounting. For 
"many of the remaining errors and omissions, one can make a 
plausible argument that these bugs could lead to that error. 
Many statements that 'we did" not account 'for were factual 
errors, for example, "Heavy rainfall occurs only in warm 
area^s." . (Heavy rainfall occurs in many cool and cold areas 



For example , . 



(The standard 



of \he world.) * Others. were naming errors. 
"When water evapo rates , it turns to steam. 

term in meteorology for the produ^ of evaporation is water 
vapor . ) • V 

■ '■■ * V \ - ' ' ' ^ 

'Many of the bugs we;^observed are -specific to the' domain 
of rainfall. This sho\ald be neither surprising nor 
disturbing. One of the skills a good teacher^ must pos'sess 
is knowledg'e. of the types of misconceptions that arise in 
■the domain^a'ught. It is likely that there- are other bags 
which occur fn students' knowledge about rainf all r ^ut.. i t ^ 
will surprise us if this number is unmanageably large. 
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e. set of observed misconceptions. 



^ M isconc ept ion' - 
(1 X "Cool in 



1 x uoo 



>"y-co;n^t.act 



Number of 3ubjee4:.s 



'Ex amp-1 e. 



(2) Heating-by-radiation 



(5) Sraall-moistiir e-source 



(4^) 'Rising-c^usesQ ^ 
iricr eased- pre ss;;re 

(5) Absorbtion^by- 

expansion^* r) 

■ : -V ■ 

C6) Heating-by-contact . 

(7) Squeezing-causesJ- 
* condensation 

((8 ) ,Temperature-of-water- 
irr elevant-f or- ' 
evaporation 

(9 ) Teinperatur e- 

dif f erential- 
causes- evaporation 

(10) Insuf f icient- 
warming-of- 
water 



, .( 1 1 ) Heat ing-causes- 
condensation 

(12) _ Winds-cause- 

_ . pressure-increases 

(13) Cool ing-c'auses- 
. evaporation 

(14) Rising- results- in 
pressure-equalization 



(15) Cooling-cause-air 
to-rise - 

(16) Evapor at ion-^causes- 
air-to-rise- 
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"Mountairfs cause condensati'on . 
because"^pld land toucj^^ng. air 
causes -co^ndensation • ^j.^^ 

"The sun' warms ^ the air." 

."^ 'T2 by 12 by 10 .foot por^d is . 
enbugh' to cause rainfall 

/ ■■ ■ ■ .. : \ - 

Rising air makes the moist air 
rise , pr essur e increa^ses . . • . " 

H". .'.decrease in pressur'e caused* 
water molecules to expand, 
causes evaporation." 

"...land war-ms the ^air at night." 

"Putting pressure on air ^ 
masses causes cpndensat ion • " 

"Temperature of water is 
unrelated to evapor aiion . " ' . 



"Air has to be cooler than 
the body of water for • 
evaporation to occur ."- 

"A current' can be warm\because 
it comes from a^arm source 
of waters — for ex ample , a ^ 
lake wliich is warm." ' ' 

"Airr warming up causes rainfall 



"Winds are for ceful and 
cause various air pressures . " 

"When a body of- water is cold, 
it evaporates . "V - ' 

"Air that is -warmer is ex pand-ed 
and has less pressure. It rises 
until its pressure is equal to 
surrounding -air." 

' - ' ' ' ■ 

"Cooling causes 'air to rise." 
, » • • ■ . • 

ft ■ 

"Evaporation cause? air to rise." 



Tb'rbpre^ent these bugs in a way that makes' it possible- 
for a* .t^achingi system with* a- d iagnose-and-correct goal 
sti^ucture to use-them is an important step.. In principle , 
the current script-like * fonrialism^ could, be used. In 

> . ■ V ■ * ^ • '■ .-. , ; , .. . .\ . 

practice^ such things as incorrect functi'bnal relationship 
• (e.g. the Heat fng— causes— condensation bu^) or i'^ncorrect 
attributes (e.q.^ the- temperatiire of mountains in the 
Cooling-by-contact bug) seem to^ require a different 
repr^entational viewpoint \than those^ provided by script 
structures.^ We will provide some steps toward- a: solution 
below/in the section on repr esentat ion*. ^ " ' " 

. Expl aining^ physical^ processes^ . The thi td problem we- 
described in the introduction i.s the nature of the 
abstractions and viewpoints ^hat, tutors use^ tQ- describe 
.physical processes. The te-Sching dialo.gues we hav^e examined 
require . a , niul ti-leveled structure^ with soript-like 
knowledge necessary to support some parts of the discussions 
and relatively low-level detailed knowledge about physical 
principles necessary ^ t*o support other parts. More 
-Interestingly r /to adequately support the dialogues requires 
that • the knowledge be factorable in several vways . Tutors 
discuss far more than causal and temporal- linkages betweerj 
^steps in a script structure. , They probe and discuss 
information about attributes of the actors that ^are. 
important^ the results of processes and the form of the 

o , t ' ' . -17- ' ■. ' r 
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^yincti^Dn^l ^relationship) which ho Ids -feeirween the attributes 
^^of^actars and the results of processes, . 

Some example^ of tuto'rs' statements and questions aVe 
shown in Table 3*. In each aase, the question or. statement 
refers t9 one of the specific aspects of the knowledge we 
just described, A cursory examination of our'dialogues 
suggest that a large percentage of tutors' statements and 
Questions- fall into-" these categories. For example, in a 
. representative'^ dialogue which* consisted of 41 exchanges^ 

. - . - . ^JF ■ ^ 

between the tutor and student, four of the- tutor's 



statements were about attribute's :^of "%^tors , £our were about 
results of processes and seven were about functional 
relationships. .This accounting includes 15 of the tutor's 
statements. Of the remaining 26, eight are references to 
priar/ intermediate or subsequent processes at a level of 
abstraction that can be handled by script, structures. The 
remainder, which we do not have good ideas about, include 
references to" the spatial structure of the processes, 
descriptions of physical principles and explication of a 
metaphor . . 

Representing the knowledge domain 




For each of the three problems .we have discussed, a key 
element necessary - for its solution ..is. an adequate formalism 
for representing the knowledge taught. To_ specify the goal 



Table -3. Example statements, for each :part of the representation. 



Factors (Attributes of Actors) 



-7 



"Do you think the\amount of moisture in the air affects the 
amount of rainfall?". " . • 

- "Does the temperature of water affect evaporation?"''' 

\ ■ • * . ■ _ 

Results of Processes 

"Condensation is the process by which moisture in the air 

becomes liquid water '3^^n." ^ . ^ 

"Evaporation is the process by which water in the ' ocean 7 
becomes moisture in the air." ' ' ' ^ 

Functional Relations . 

"What happens to the temperature -of the air as it rises?" 

" * . 

"Do you remember . how temperature affects' evaporation?" 



/ 
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structure of a tutor., in detail, requi^5<es specifying 
misconceptions and methods for correcting them. Specifying 
misconceptions and the proper abstractions and viewpoints 
from which to diagnose and correct them requires a detailed 
formalism for representing the knowledge taught. Script 
structures can be- used to- represent ordered causal and 
temporal processes, ^but this handles only a small number of 
-bug representations and viewpoints from" which to discuss 
^ them. In this section, we will describe^ one additional 

representational viewpoint, that' .seem^ important for a 

" ■■■ • . ■ ■ v" . • \ ■• . 

tutorial system. ' -. 

A representation ^of functional relationships . Th e 
basic' unit of our representation for functional 
rela^tionships is. a description of some process such as 
^ ; cooling or evaporation. .An example is shown in Figur-e 2. 
This represents the process of evaporation as i't. occurs in 
the rainfall domain. Its parts are: 

(1) -?|^^set of actors each with a role in the overall 
^ process. For example.^ the ocean plays the Tole of 

moisture source. ^ 

(2) A set of factors which affect- the process. The fa^ctors 
are .all attributes _of' actors. 'For example, the 
temperature of the source , body-of-water is a factor in 
evaporation. - . 

o - ■ _ ^ ■ 
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yre 2. A -fjundtionaL representation for evaporation- 

. . > "■• ' ' ' ■ ' ■ . ■ •■■ 

EVAPORATION :r . ' . ^. ' 

- Actors 

Source: Large-body^of-water 
Destinat iQn :- Air-mass . •. 

Factors - " • , ' * \ 

Temper aiureC Source) 
Temperature (Destinat ion) ' * 
Prox imityCSpurce , 'Destination") . . - 

Pun c t ion al- relation ship 

Positive(Temperature(Source) ) 
PositiveCTemperatureCDestination) ) 
"Positive(Projcimity(Source , Destination)) 
Result . ^ , / •. 

In'creaseCHumid ity(Destination) ) * ; 



(3) -^A description of the result of the process. The resdlt 

is always, a change of an 'attribute of one of the 
^ actors. For example f - th'e result of evaporation is to 

• increase the humidity of the destination air mass, 

■ ■ ■ ■ ' ■ _ ; - ■ 

(4 ) . A :descr ipt id-n of the functional relationship which 

holds among the factors and the' result. -We. believe 
•-'" there is room for complexity and subtlety in the 

description of functional relations , but we currently 
'use a simple descriptive scheme which allows positive 

sand inverse" relationships . For example, in evaporation 
-there Is a positive relationship between the, 

temperature of the moisture source and the resulting 

humrdity of the air mass* 

This representation is general in two ways. It can- be 

partially 'specified by. a.ssigning values to . the actor 

' ' . ' ■ '■ .. . . , 

attributes.. For example, representing " an instance of a 

large amount of evaporation requires assigning values like 

"warm" to. the temperatures of sotirce ana destination and a 

value like, "adjacent" to" the • proximity ' r'elat ionship 

Inference rules which make use of the iPnformation about 

relevant attributes and functional relationships can be 

constructed • to check C^t some level of^approximation) if the 

assigned values of factors and result are consistent. 
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The second way that this representation can' be further 



specified is by instantiating the actors.. For example/ in 

the case of rainfall over Ireland, the source is^the Gulf 

\ ■ . ■■ ■ ■ ^ ■ 

Stre^am* 



ThuSr the information in this representatioiji provides a 
way for generating representations of different amount^s of 
evaporation and for representing these different -amounts 
with different actors^ ^ This representation provides" an 
additional representational viewpoint that^ is missing froDj. 

script structures. ' - 

■ ' • ' • ■■• ' . ' ( ■", ■-' ■■ . ■ ' .'^>^ 

Re pr e se n t i hg Bug s ♦ In ^addition to representing 



knowledge that is correct/ it must be poss to represent 



.misconceptions. One. constraint' oh a knowledge 
repr^esentation used for teaching.' is -to represent 

1 

■.misconceptions as meaningful transformations of the basic 
knowledge representations (Brown and ,Burton, 1978). For 
example r consider- the representation for the 
Absorption-by-expansion bug shown in Figure 3. The key part 
is highlighted. It consists principally of a substitution 
of pressure for temperature as the relevant attribute of the 
destination irt the normal ^ representation for evaporation^ 
Representing bugs 'in the same format as the correct 
knowledge makes differential- diagnosis possible. In trying 
to"'" decide whether a student has the cool ing-By-contact bug . 



Evaporation under the Absorpt iori-by— exp^ansion bug 
Actors ~ . 

• . Source: Large'- bod y-of-water 

Destination : Air-mass 
Factor s ' ■ 

P2?essu2'e ( Dest ination) 
PrpximityC Source , Destination) 
Func tional-rel at ion ship 

'Positive (Prox imityC SojS^ce, Destination)) 

, ■■'...I- ■ _ ■ ' ' — 

Result ■ ^/^ 

Increase (Humid it yC DeTsti nation ) ) 



asking the student- wjhat actors are involved will hot provide 
ariy relevant, information. Either winds, mountains or; cold 
air - masses will S'till b.e mentioned as important . . I^: is the 
role or attributes of these actors that supplies ^ the 

r ■ r - ■ ' _ 

■ . ■ ■. ^ ■ . • ^r" • , . . 

leverage However, > in' the Smal 1-moisture— soxirce bug^ the 
actor itself suppl ies the diagnostic leverage point . ' . - 

Bugs show up in all parts of the representation. For 
example, the Cool ing-by-contact bug is represented as a_ 
difference in the r.ole of the object, or as a diffefrence in 
the relevant attribute of ^the object. 'The 
Heating— causes— condensation 'bug is represented *^s a 
difference in ^ functional r^el at ionship . . .The 

^Smal 1-moisture-source^ bug i.s represented as a difference of 
actor in the source role . V 

Rem airing Problems - . 

■ - ^ ' . . ■' ' ' 

^ In the previous sections, we outlined some tentative- 
Steps -toward splving the problems of goal structure, 
representing -misconceptions, ' and .provid ing the. additional 
representatibnal viewpoint* of a set of functionally related 
processes. We believe the heart of these problems^ 1 ies in 
the representation of knowledge.^ Our tentative steps toward 
representing knowledge and misconceptions about "physic'al 

- ■ . ( -i ■ * ■ ■ ■ : ^' ^ 

procesises extend the^ script-like representation we have rbeen 
using, but we believe we are "^still just scratching th,e^- 



surface . , , 
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'' Adding "this one .viewpoint has given us more windows 
into -.problems that were opaque using our previous notation. 
In. the -remainder of this F>aper> we will point out the nature 
of some of the problems that we can now see. • 



■ Interacting Bugs ^ In most cases, a single bug accounts 
for each error , but there are cases where bugs interact to. 
produce a single^ misconception* Brown and Burton (19<78) 
have shown that in arithmetic, students often have a set of 
bugs that ;- interact to produce non-obvious pa-tterns of 
errors. The observations from our experiment . suggest that 
similar things happen in the rainfall domain* For example^, 
one subject said in response to a question about, the role of 
cold 411^ masses: 



"Cold air masses hitting warm air masses cause 
condensation." 



Since she mentioned contact -^nd not rising, the' most 
Straightforward diagnosis from this statement is; that she 
has a Cooiing-by-contact bug;- However, -the problem . real ly 
seems ; to be d ue to ^ two ' iniier act ing' . bugs : ' the 
Heating-by-contact bug^^ and the Heat i.ng-causes-condensat ion 
bug. Two of this student' is .responses -to other questions 



"Air warming up causes rainfal 1 • " 
"Tropical wiiids warm air." 



Thus , her description of condensation caused by . cold air 
masses hitting warm air masses is most likely due to these 
. bugs interacting to produce a model in which the cold air is 
warmed up from contact with the warm-^ air (the 

,N ■ ' ' 4 ' ' . ■ 

Heating-by-contact bug) and this warming ;causes rainfall 
(the Heating-causes-condensation bug). ^" 

The existence of interactions im.pl y^ that- the- -mapping 

from errors to bug^ is not one-torone. We suspect that 

•* *- 
there are many cases .where the relatibriship between 6 set of 

errors and the underlying bugs may be quitfe .s\jbtle. 

""existente.^^o^^ no^n-obviqus^ account. \for v.ouV 

inability ^o, classify many of the errors we observed. 

Wh.ere do bugs cbme from?- * Having Oooked at th§ bug's we 
have isolated^ we now believe that they still are relatively 
shallow^ reflecting even deeper levels of misconceptions in 

^ - - • » ■ 

knowledge. The major reason for belffeving this is that bugs 
themselves „ seem to - form patterns. .The patterns seem^best; 
explained as the result of deeper problems in the student's 
knowledge. Sometimes these deeper problems are due to the 
.application of an incorrect" metaphor in understanding a 
process; other times ,.the patterns reflect incorrect or 



m'is-sing general . relationships between process, like the 
notion of inverse, or, positive feedback* 

An example of a pattern that reflects an incorrect 

. . •• ■ ■ ■ • 'O' -..^ • • 

applicatiort of a metaphor is the "sponge .pattern".. It 
- includes > two bugs : the Absorption-by-expansion bug arid the 
Squ€feZihg-cai^es-^condensat ion bug. In effect, the student 
views the air. maSs as. a giant sponge, expanding ^o" absorb . 
moisture and later having it squeezed out. Tutors typically 
deal witii this deep^level yraiscoTiceptio'n by using a 
"container" analogy . for^ the| air mass, identifying, the 
"capacity of the con£ainer with air temperature rather than 
size. ■ ' ' ■ . - ^ 

A second type of pattern is that which arises because 
■ of m i ss ing g en e r a 1 i z a ti 6ns abd u t • pr ofr^ ss r el a t ionsh Ips . ■ Fo r 
• example, the pattern , which includes the 

f * . ■ . _ ^^'^ 

H-eating-causes-condensation bug but which also includes - the _ 

■ ■ - ■ . . ' ' 

correct functional relationship between he^'ating . and - 
.evaporation; seems - to reflect" the student's lack of 



understand ing that condensation and evaporation are inverse 

;■ .' X / - ., .. . . ^ \.. .... ..^ . .... . - . • • 

process-^s . ^ Tutors Qeal with : this' bugj jDy irif.onning' the; 



s1:udent that, the two processes are inverses and.^ expl aining 
the sense in which they are inverses. 

These "processes of understand ing draw on a large set of 
real-world '■ knowledge that students have built up over their- 



lifetime • The bugs often seem to depend on the student's 
failure to understand some deep physical principles that 
support the correct model. In order for tutors to deal with 
conceptual bugs, they must recognize this- mode of- 
understanding and attempt to discover what models the 
Student applies to understand the processes taught. 

Other representational viewpoints . The existence of 
patterns of bugs implies that that there are still other 
representat iorv^'l viewpoints necessary^ to deal completely 
with physical processes. The analogical use of the "sponige" 
concept* implies that a complete analysis will require 

techniques for- representing and mod-ifying models-drawn from 

■ ^ ■ * ■ ' • ' • " 

other domains. The process-relationship example implies' 

that representation of . .general^ process . relationships like 

.".inverse processes" y ; \"f.eedb 'system" ,; /and \_ "cyclical 

process" will* have to be included. There also §eem ''to .be^ 

multiple ways of describing what appears to be essentially 

the same information* These different . ways . may- be,, 

generative in . nature^ but they emphasize different aspects 

of the processes.- .For example^ there is* the - energy 

viewpoint from v^hich various processes appear to add or 

remove types- of energy to dif fereht actdrs. . There is the 

■■ " . ■ - ■ ^- . " ■ ■ * - c . ... . 
change— of-state ■ /viewpoint^ from wtifch-^ various actors appear 

to change form and location as time progresses. Thes^ 

* r - ' * • . ■ , , ■ ' '■ - 

multiple representational' viewpoints are different but must 



interact in ord^r to provide a • cpiBplete {cepresentation .of a 
physical process. We t^^eli'eve- -that.A defining- them will 

provide additional insights into the nature of tutorial 

■ - ■ 4 ■ • ■ , ' . ' . ' ' 

skill . • ' - . ' ' 
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